Regenerative telemetry method for resource reduction

ABSTRACT

A system and a method for interpreting normal and errored segments of network measurement data differently, and processing network measurement data, including: receiving a first query for network measurement data; extracting compressed network measurement data and error metadata from a repository database based upon the first query; decompressing the extracted compressed network measurement data; retrieving an error segment of raw network measurement data based upon the error metadata; and merging the error segment of raw network measurement data with the decompressed extracted compressed network measurement data to produce extracted network measurement data.

TECHNICAL FIELD

Various exemplary embodiments disclosed herein relate generally toregenerative telemetry method for resource reduction.

BACKGROUND

Optimizing network operations and particularly network performancerelies on measurements such as key performances indicators (KPI) acrossthe network, and monitoring these measurements on regular basis based ona policy, such as at every 15 minutes. Monitoring measurements fromremote nodes is referred to as telemetry in general.

The raw measurement data collected at remote network sites are typicallysent to a central computing platform on regular basis for archiving andanalysis.

The data generated by network equipment may indicate normal systemoperation or it may indicate an error condition or an anomaly. Whenthere is an error indication, the analysis procedure triggers networkoperations functions to identify the reason of the error and otherfunctions to eliminate the root cause of the error.

At the whole network level, there will always be several errorindications reported by the fault management system, but the networkequipment and links are designed, configured, and run in such a way thatthe error conditions are rare, and the network reliability stays wellabove 99%.

SUMMARY

A summary of various exemplary embodiments is presented below. Somesimplifications and omissions may be made in the following summary,which is intended to highlight and introduce some aspects of the variousexemplary embodiments, but not to limit the scope of the invention.Detailed descriptions of an exemplary embodiment adequate to allow thoseof ordinary skill in the art to make and use the inventive concepts willfollow in later sections.

Various embodiments relate to a method of processing network measurementdata, including: receiving a first query for network measurement data;extracting compressed network measurement data and error metadata from arepository databased based upon the first query; decompressing theextracted compressed network measurement data; retrieving an errorsegment of raw network measurement data based upon the error metadata;and merging the error segment of raw network measurement data with thedecompressed extracted compressed network measurement data to produceextracted network measurement data.

Various embodiments are described, further including receiving, by therepository, the compressed network measurement data and the errormetadata for a network object.

Various embodiments are described, further including compressing networkmeasurement data and transmitting the compressed network measurementdata to the repository.

Various embodiments are described, further including receiving, by amonitoring system, the error segment of raw network measurement datafrom a network object.

Various embodiments are described, further including detecting an errorin the network measurement data and sending only the error segment ofnetwork measurement data to the monitoring system.

Various embodiments are described, wherein detecting the error includesfiltering the network measurement data using thresholds.

Various embodiments are described, further including: receiving a secondquery for network measurement data; extracting compressed networkmeasurement data and error metadata from a repository databased basedupon the second query; decompressing the extracted compressed networkmeasurement data from the second query; determining that thedecompressed network measurement data from the second query has aspecified accuracy; and outputting the decompressed network measurementdata from the second query as the extracted network measurement data.

Further various embodiments relate to a telemetry system, including: arepository module configured to receive compressed network measurementdata and error metadata; a query module configured to receive a queryfor network measurement data and to extract compressed networkmeasurement data and error metadata from a repository database basedupon the query; a decompression module configured to decompress theextracted compressed network measurement data; an error retrieval moduleconfigured to retrieve an error segment of raw network measurement databased upon the error metadata; and a fusion module configured to mergethe error segment of raw network measurement data with the decompressedextracted compressed network measurement data.

Various embodiments are described, further including a monitoring systemconfigured to receive the error segment of raw network measurement datafrom a network object.

Various embodiments are described, further including a compressionmodule configured to compress the network measurement data and transmitthe compressed network measurement data to the repository.

Various embodiments are described, further including an error detectionmodule configured to detect an error in the network measurement data andsend only the error segment of network measurement data to themonitoring system.

Various embodiments are described, wherein the error includes filteringthe network measurement data using thresholds.

Various embodiments are described, further including a compressionmodule configured to compress the network measurement data and transmitthe compressed network measurement data to the repository.

Further various embodiments relate to a non-transitory machine-readablestorage medium encoded with instructions for processing networkmeasurement data, including: instructions for receiving a first queryfor network measurement data; instructions for extracting compressednetwork measurement data and error metadata from a repository databasedbased upon the first query; instructions for decompressing the extractedcompressed network measurement data; instructions for retrieving anerror segment of raw network measurement data based upon the errormetadata; and instructions for merging the error segment of raw networkmeasurement data with the decompressed extracted compressed networkmeasurement data to produce extracted network measurement data.

Various embodiments are described, further including instructions forreceiving, by the repository, the compressed network measurement dataand the error metadata for a network object.

Various embodiments are described, further including instructions forcompressing network measurement data and transmitting the compressednetwork measurement data to the repository.

Various embodiments are described, further including instructions forreceiving, by a monitoring system, the error segment of raw networkmeasurement data from a network object.

Various embodiments are described, further including instructions fordetecting an error in the network measurement data and sending only theerror segment of network measurement data to the monitoring system.

Various embodiments are described, wherein detecting the error includesfiltering the network measurement data using thresholds.

Various embodiments are described, further including: instructions forreceiving a second query for network measurement data; instructions forextracting compressed network measurement data and error metadata from arepository database based upon the second query; instructions fordecompressing the extracted compressed network measurement data from thesecond query; instructions for determining that the decompressed networkmeasurement data from the second query has a specified accuracy; andinstructions for outputting the decompressed network measurement datafrom the second query as the extracted network measurement data.

Further various embodiments relate to a telemetry system, including: arepository means receiving compressed network measurement data and errormetadata; a query means for receiving a query for network measurementdata and to extract compressed network measurement data and errormetadata from a repository database based upon the query; adecompression means for decompressing the extracted compressed networkmeasurement data; an error retrieval means for retrieving an errorsegment of raw network measurement data based upon the error metadata;and a fusion means for merging the error segment of raw networkmeasurement data with the decompressed extracted compressed networkmeasurement data.

Various embodiments are described, further including a monitoring meansfor receiving the error segment of raw network measurement data from anetwork object.

Various embodiments are described, further including a compression meansfor compressing the network measurement data and transmit the compressednetwork measurement data to the repository.

Various embodiments are described, further including an error detectionmeans for detecting an error in the network measurement data and sendingonly the error segment of network measurement data to the monitoringsystem.

Various embodiments are described, wherein the error includes filteringthe network measurement data using thresholds.

Various embodiments are described, further including a compression forcompressing the network measurement data and transmit the compressednetwork measurement data to the repository.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to better understand various exemplary embodiments, referenceis made to the accompanying drawings, wherein:

FIG. 1 illustrates the high-level architecture of an embodiment of thenetwork telemetry system;

FIG. 2 illustrates a flowchart for frontend module processing of one KPImeasurement; and

FIG. 3 illustrates a flowchart for backend module processing of onequery.

To facilitate understanding, identical reference numerals have been usedto designate elements having substantially the same or similar structureand/or substantially the same or similar function.

DETAILED DESCRIPTION

The description and drawings illustrate the principles of the invention.It will thus be appreciated that those skilled in the art will be ableto devise various arrangements that, although not explicitly describedor shown herein, embody the principles of the invention and are includedwithin its scope. Furthermore, all examples recited herein areprincipally intended expressly to be for pedagogical purposes to aid thereader in understanding the principles of the invention and the conceptscontributed by the inventor(s) to furthering the art and are to beconstrued as being without limitation to such specifically recitedexamples and conditions. Additionally, the term, “or,” as used herein,refers to a non-exclusive or (i.e., and/or), unless otherwise indicated(e.g., “or else” or “or in the alternative”). Also, the variousembodiments described herein are not necessarily mutually exclusive, assome embodiments can be combined with one or more other embodiments toform new embodiments.

Many objects in the network (applications, function modules, hardware,virtual machines, communication links) continuously generate measurementdata. Because there are thousands of network equipment and communicationlinks in the network, the transmission, collection and storage of themeasurement data quickly becomes unmanageable. To bound the scale of theproblem, typically, the raw measurement data is summarized for 5, 15,30, or 60 minute intervals, which results in significant loss ofinformation.

Because of these resource constraints (for transmission of the data andstorage of the data), communication systems generally do not attempt tocollect highly granular measurements, such as at 1 second or 100millisecond resolutions. On the other hand, as the industry is movingtowards highly intelligent, autonomous networks with closed loopcontrol, availability of measurement data from the network nodes insmall timescales are becoming more and more critical. By using theexisting solutions, it is not possible to make such large amounts ofmeasurement data available for the cognitive control of the network.

To overcome the resource intensive scaling problem of network telemetry,existing solutions take one or more of the following approaches.

In a first approach, data is summarized over an interval, for example 5,15, 30, or 60 minutes. This approach is an integral part of alltraditional network systems. This approach results in loss ofinformation.

In a second approach, data is compressed before being sent to a centralrepository. Popular monitoring systems provide the capability to workwith compressed data—using lossless compression. The losslesscompression ratio of highly random network data will be verysmall—typically less than 2:1 as compression algorithms rely on patternsand asymmetries in data symbol rates. As a result, the savings in theamount of data transmitted and stored is not as much as would bedesired.

In a third approach, data is stored on the network equipment and notcollected regularly. This solution will consume storage resources on thenetwork equipment. This is not feasible in many applications such asinternet of things (IoT) and other low resource situations. The solutionwill limit the capability to detect and resolve errors. This approachwill constrain access procedures for analysis, and the data retrievalstep will utilize network bandwidth.

In a fourth approach, data processing functions are pushed to thenetwork equipment instead of pulling the data from the equipment. Thisapproach consumes substantial compute and storage resources on thenetwork equipment. Distributed application management of the applicationagents create design and operation complexity of the network equipment.Further, analysis of data from multiple network equipment becomes morecomplex.

In a fifth approach, using a hieratical structure, data can be collectedat distributed nodes, then transferred to the central analysis unit asneeded. These hieratical nodes may be network element managers orsimilar functional modules. In this approach, the data will not bereadily available for analysis, but it must be collected from thedistributed nodes upon a query. Management of the distributed nodes addsa level of complexity to the system design and operation, and the dataretrieval step will utilize network bandwidth.

The embodiments of a network telemetry system described herein are basedon the intuition that most of the telemetric measurement data collectedin the network is part of “normal” operation of the network withouterrors that need to be acted on. The network telemetry system based onthe invention works as follows. The network measurement data iscollected at fine granularity at or within the proximity of the sourcenode. The network measurement data is passed to a compression module andan error detection module. The error detection module analyses thetimeseries of network measurement data to identify any data segmentswith errors. If errors are detected, the data segment of the timeserieswith errors is sent to the monitoring system as raw network measurementdata. If the monitoring system supports it, the data may also be sentusing lossless compression. The monitoring system may be one of exitingnetwork operations modules that stores and provides network measurementdata using various pre-existing methods.

The error detection module also sends information about the errorsegments to the compression module as metadata. It should be noted thatthe error detection criteria applied at this stage is rudimentary—it isnot expected to be at the level of complex anomaly detection, which istypically executed as post-processing at central computing platforms. Asa result, the computing load at network elements is minimal. Thecompression module may apply lossy compression on the raw networkmeasurement data passed to it for a given time interval—such as for 1day. Then it packages the compressed network measurement data togetherwith the metadata indicating the error segments. Then the compressingmodule sends the package to the central repository.

It should be noted that because lossy compression techniques are used,the amount of data sent to the repository has significant size reductiondespite having randomness—which typically cannot be compressed bylossless compression algorithms.

At query time, a user requests data for a measurement interval from thequery module. The query module regenerates the network measurement databy decompressing segments of compressed network measurement dataretrieved from the repository.

From the metadata within the package that is retrieved from therepository, the query module checks if there were any data segments witherrors in the measurement interval requested. If there were errors, thenthe query module accesses the monitoring system to retrieve errorsegments, then it overlays the error segments over the decompressedtimeseries of network measurement data reconstructed from therepository. If regenerated network measurement data is accurate enoughfor detecting anomalies as determined by the user, then the steps forretrieving and overlaying raw network measurement data of the errorsegments may be eliminated. This means that the overlaying of the rawnetwork measurement data only occurs as needed by the requirements ofthe user.

The user executing the query may be a network operations administratoror an autonomous cognitive module or any other autonomous system. Thenetwork measurement data archived and retrieved via this process isexpected to be used for network performance optimization, businessoptimization, or any other high-level analytics processing.

FIG. 1 illustrates the high-level architecture of an embodiment of thenetwork telemetry system. The telemetry system 100 has a frontend module110 that is co-located with a network object (not shown), which producesthe input network measurement data 105. The implementation of thefrontend module 110 may take many forms. The frontend module 110 may beintegrated with the network object as a library, or it may be integratedwith the network object as a hardware element, including ageneral-purpose compute node within the proximity of the networkobject—to consume minimal network bandwidth for their interaction. Ifhigh performance is required to collect the network measurement data,then a specific hardware solution, using for example field programablegate arrays (FPGA), may be used as well. The frontend module 110 mayserve one or many network objects within its vicinity.

The input network measurement data 105 at the frontend module 110 mayinclude identifiers such as key performance indicators (KPI) (k_(i)) andauxiliary variables (a_(i)). Auxiliary variables indicate informationsuch as the timestamp of data origination, its duration, and measurementunits.

The frontend module 110 includes an error detection module 115 and thecompression module 120. The error detection module 115 processes theinput network measurement data 105 in real-time. If the error detectionmodule 115 detects an error in the input network measurement data 105,then the error detection module 115 sends the segment of the inputnetwork measurement data 105 that includes the errors to a monitoringsystem 130. It also sends information about the error segments in theform of metadata to the compression module 110.

The error detection module maintains a first-in-first-out buffer to holda small amount of the input network measurement data 105. When an erroris detected, not only the error but also the small amount of networkmeasurement data prior to the error is also sent to the monitoringsystem to provide the capability to better analyze the events leading tothe error. This buffer size would be adjustable. upon the capabilitiesof the network object or other object implementing the frontend module110 and the overall goals and performance requirements for anomalydetection.

The error detection 120 module is not expected to execute an elaborateanomaly detection procedure. It rather applies a broad filter on theerror condition. It is expected to have simple threshold detectionlogic. Network operations infrastructure will typically have advancedanomaly detection applications as part of the fault managementsubsystem. The definition of error condition may be determined as asystem level policy per KPI. However, considering the current trendwhere the network objects and all network equipment are becoming moreadvanced with substantial computing power, it is likely that the fullanomaly detection algorithms may be executed at some remote networkobjects. Hence, the complexity of the error detection performed by theerror detection module 120 will be based upon the capabilities of thenetwork object or other object implementing the frontend module 110 andthe overall goals and performance requirements for anomaly detection.

The compression module 115 collects the input network measurement data105 for a pre-set time interval—such as for 1 day, but other intervalsmay be used as well. At the end of the interval, the compression module115 performs lossy compression on the input network measurement data105, and the frontend module 110 then packages the compressed inputnetwork measurement data and any metadata received from the errormodule. Then the frontend module 115 sends the package 125 to therepository module 140.

The monitoring system 130 receives and stores segments of the inputnetwork measurement data 105 that includes the errors. This data maythen later be accessed by a backend module 135. The monitoring system130 may reside on the network where it is most convenient and efficient.The monitoring system 130 may be part of a central computing platform(not shown) or it may be a stand-alone system. The monitoring system 130may include storage for storing segments of the input networkmeasurement data 105 that includes the errors and a controller thatallows for access to the data stored.

The backend module 135 that is typically part of the central computingplatform may include a repository module 140, a database 145, a querymodule 150, an error retrieval module 160, a decompression module 165,and a fusion module 170.

The repository module 140 typically resides at a central computingplatform (not shown). The repository module 140 stores incoming datapackages 125 in a database. The repository module 140 may includestorage of any type and a controller that accesses the stored datapackages 125.

To access the saved network data, a user enters a query 155 to the querymodule 150. As noted above, the user can be person (e.g., a networkmanager or technician) or an automated system. The query 155 includesKPI identifiers and auxiliary variables indicating the time and theduration of the data (the auxiliary variables may also include otherproperties such as the format of the data.) The query 155 will alsoidentify the specific network elements for which data is requested.

The query module 150 accesses the database 145 to retrieve packages 125corresponding to the given time interval. Then the query module 150passes the compressed data to the decompression module 165 and checks ifthe metadata indicates any error segments. The query module 150 passesany error segment information included in the package 125 to the errorretrieval module 160.

The decompression module 165 recovers the original network measurementdata according to the identified accuracy settings and passes thedecompressed data to the fusion module. The error retrieval module 160accesses the monitoring system 130 to retrieve raw network measurementdata for the error segments based upon the error segment informationreceived from the query module 165. Then the error retrieval modulepasses the retrieved raw network measurement data to the fusion module170 if there are any error segments in the given query interval.

The fusion module 170 merges together the decompressed networkmeasurement data and the raw network measurement error data. If thereare any raw network measurement error data received from the errorretrieval module 160 in the duration of the decompressed data, thefusion module 170 replaces the corresponding segments of thedecompressed network measurement data with the raw network measurementerror data. The fusion module 170 then outputs the data as onetimeseries. It also provides a metadata block that indicates to the userincluding the statistics of the accuracy of the compression process andthe location of the error segments if any.

In another embodiment, if requested from the query module, the telemetrysystem 100 can trigger immediate generation and transfer of pending data(normal or error) from the frontend modules 110.

FIG. 2 illustrates a flowchart for frontend module processing of one KPImeasurement. The frontend module 110 receives input network measurementdata 205 for a KPI (k_(i)) over a specific timeframe of duration t. Thefrontend module 110 compresses the data, step 215. The frontend module110 also detects if there if any errors are found for the KPI during thetimeframe. If an error is found in the KPI measurement, the frontendmodule 110 sends the raw network management data to the monitoringsystem 130. Also, at this point metadata indicating the presence of anerror is produced and sent to be packaged with the compressed data. Thefrontend module 110 then packages the compressed network measurementdata and the error metadata, step 225, and sends the package, step 230,to the repository module 140.

FIG. 3 illustrates a flowchart for backend module processing of onequery. The backend module 135 receives the query 305 which may include atimeframe including start date and time and a time duration. Further,the query may indicate the network object for which network measurementdata will be searched. Next, the backend module 135 will retrievepackage(s) from the database 145 corresponding to the query. The backendmodule 135 then decompresses the compressed network measurement data inthe package. The backend module 135 also determines if there are anyerror segments during the query timeframe, 320, and if so, the backendmodule 135 retrieves, step 330, the raw network measurement data fromthe monitoring system 130. The backend module 135 then merges any rawnetwork measurement data for the error segments with the decompressednetwork measurement data, step 325. The backend module 135 then outputsthe merged output data for the query 335.

Also, as part of determining if there is any error segments during thequery timeframe, step 320, the backend module 135 may determine that theaccuracy of the decompressed network measurement data is accurateenough, and if so not seek to determine if there is any error segmentsduring the query timeframe or to retrieve such data, and instead justuse the decompressed data.

Embodiments of the telemetry system may use a variety of choices toapply lossy compression to the network measurement data at the frontend.One embodiment may use deep neural network. Embodiments may also useopensource tools such as ZFP,https://computation.llnl.gov/projects/floating-point-compression. Thetool allows adjustment of the compression rate and the error magnitudeof the output. The tool has an implementation in C++ programminglanguage with a small footprint. It reaches compression throughput up to2 GB/second.

The various modules described herein will be implemented using softwareor program instructions running on a processor. In situations, wherehigh performance is required, specific hardware implementation of themodules may be used as well.

The embodiments described herein solve the technological problem ofcollecting network measurement data with enough accuracy and granularityto allow for advance network anomaly detection, and data analysis forbusiness or operations optimization purposes. Currently, network dataanalysis schemes are being developed using, for example, deep neuralnetworks, machine learning, etc., that require accurate networkmeasurement data. As described above, to collect such data uses up a lotof data storage and/or network bandwidth. As a result, existing datacollection methods may not provide the needed capacity or may result inhigh network and storage resource utilization. Thus, the telemetrysystem embodiments described herein provide a method and system thatprovides network measurement data with sufficient accuracy withoutrequiring excessive storage, network bandwidth, or hardware resources.

The embodiments described herein may be implemented as software runningon a processor with an associated memory and storage. The processor maybe any hardware device capable of executing instructions stored inmemory or storage or otherwise processing data. As such, the processormay include a microprocessor, field programmable gate array (FPGA),application-specific integrated circuit (ASIC), graphics processingunits (GPU), specialized neural network processors, cloud computingsystems, or other similar devices.

The memory may include various memories such as, for example L1, L2, orL3 cache or system memory. As such, the memory may include staticrandom-access memory (SRAM), dynamic RAM (DRAM), flash memory, read onlymemory (ROM), or other similar memory devices.

The storage may include one or more machine-readable storage media suchas read-only memory (ROM), random-access memory (RAM), magnetic diskstorage media, optical storage media, flash-memory devices, or similarstorage media. In various embodiments, the storage may storeinstructions for execution by the processor or data upon with theprocessor may operate. This software may implement the variousembodiments described above.

Further such embodiments may be implemented on multiprocessor computersystems, distributed computer systems, and cloud computing systems. Forexample, the embodiments may be implemented as software on a server, aspecific computer, on a cloud computing, or other computing platform.

Any combination of specific software running on a processor to implementthe embodiments of the invention, constitute a specific dedicatedmachine.

As used herein, the term “non-transitory machine-readable storagemedium” will be understood to exclude a transitory propagation signalbut to include all forms of volatile and non-volatile memory.

Although the various exemplary embodiments have been described in detailwith particular reference to certain exemplary aspects thereof, itshould be understood that the invention is capable of other embodimentsand its details are capable of modifications in various obviousrespects. As is readily apparent to those skilled in the art, variationsand modifications can be affected while remaining within the spirit andscope of the invention. Accordingly, the foregoing disclosure,description, and figures are for illustrative purposes only and do notin any way limit the invention, which is defined only by the claims.

What is claimed is:
 1. A method of processing network measurement data,comprising: receiving a first query for network measurement data;extracting compressed network measurement data and error metadata from arepository database based upon the first query; decompressing theextracted compressed network measurement data; retrieving an errorsegment of raw network measurement data based upon the error metadata;and merging the error segment of raw network measurement data with thedecompressed extracted compressed network measurement data to produceextracted network measurement data.
 2. The method of claim 1, furthercomprising receiving, by the repository, the compressed networkmeasurement data and the error metadata for a network object.
 3. Themethod of claim 2, further comprising compressing network measurementdata and transmitting the compressed network measurement data to therepository.
 4. The method of claim 1, further comprising receiving, by amonitoring system, the error segment of raw network measurement datafrom a network object.
 5. The method of claim 4, further comprisingdetecting an error in the network measurement data and sending only theerror segment of network measurement data to the monitoring system. 6.The method of claim 5, wherein detecting the error includes filteringthe network measurement data using thresholds.
 7. The method of claim 1,further comprising: receiving a second query for network measurementdata; extracting compressed network measurement data and error metadatafrom a repository database based upon the second query; decompressingthe extracted compressed network measurement data from the second query;determining that the decompressed network measurement data from thesecond query has a specified accuracy; and outputting the decompressednetwork measurement data from the second query as the extracted networkmeasurement data.
 8. A telemetry system, comprising: a repository moduleconfigured to receive compressed network measurement data and errormetadata; a query module configured to receive a query for networkmeasurement data and to extract compressed network measurement data anderror metadata from a repository database based upon the query; adecompression module configured to decompress the extracted compressednetwork measurement data; an error retrieval module configured toretrieve an error segment of raw network measurement data based upon theerror metadata; and a fusion module configured to merge the errorsegment of raw network measurement data with the decompressed extractedcompressed network measurement data.
 9. The telemetry system of claim 8,further comprising a monitoring system configured to receive the errorsegment of raw network measurement data from a network object.
 10. Thetelemetry system of claim 9, further comprising a compression moduleconfigured to compress the network measurement data and transmit thecompressed network measurement data to the repository.
 11. The telemetrysystem of claim 8, further comprising an error detection moduleconfigured to detect an error in the network measurement data and sendonly the error segment of network measurement data to the monitoringsystem.
 12. The telemetry system of claim 11, wherein detecting theerror includes filtering the network measurement data using thresholds.13. The telemetry system of claim 11, further comprising a compressionmodule configured to compress the network measurement data and transmitthe compressed network measurement data to the repository.
 14. Anon-transitory machine-readable storage medium encoded with instructionsfor processing network measurement data, comprising: instructions forreceiving a first query for network measurement data; instructions forextracting compressed network measurement data and error metadata from arepository database based upon the first query; instructions fordecompressing the extracted compressed network measurement data;instructions for retrieving an error segment of raw network measurementdata based upon the error metadata; and instructions for merging theerror segment of raw network measurement data with the decompressedextracted compressed network measurement data to produce extractednetwork measurement data.
 15. The non-transitory machine-readablestorage medium of claim 14, further comprising instructions forreceiving, by the repository, the compressed network measurement dataand the error metadata for a network object.
 16. The non-transitorymachine-readable storage medium of claim 14, further comprisinginstructions for compressing network measurement data and transmittingthe compressed network measurement data to the repository.
 17. Thenon-transitory machine-readable storage medium of claim 14, furthercomprising instructions for receiving, by a monitoring system, the errorsegment of raw network measurement data from a network object.
 18. Thenon-transitory machine-readable storage medium of claim 17, furthercomprising instructions for detecting an error in the networkmeasurement data and sending only the error segment of networkmeasurement data to the monitoring system.
 19. The non-transitorymachine-readable storage medium of claim 18, wherein detecting the errorincludes filtering the network measurement data using thresholds. 20.The non-transitory machine-readable storage medium of claim of claim 14,further comprising: instructions for receiving a second query fornetwork measurement data; instructions for extracting compressed networkmeasurement data and error metadata from a repository database basedupon the second query; instructions for decompressing the extractedcompressed network measurement data from the second query; instructionsfor determining that the decompressed network measurement data from thesecond query has a specified accuracy; and instructions for outputtingthe decompressed network measurement data from the second query as theextracted network measurement data.